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(57) Abstract 

A three-dimensional video dis- 
play for displaying moving images in- 
cludes a screen (4) on which succes- 
sive frames of holographic images are 
displayed, a coUimated light source (6) 
that can be from a single direction or 
scanned repeatedly through a series of 
discrete directions onto the screen in 
synchronism with the scanning of suc- 
cessive frames, the arrangement being 
such that a three-dimensional moving 
image can be observed on the screen. 
The image screen (4) may be an opti- 
cally addressed spatial light modulator 
(OASLM) and may include front and 
rear plates (8, 9) containing a layer of 
a ferroelectric or nematic or long pitch 
cholesteric liquid crystal material (13) 
and a layer of a light sensitive layer 
(15). Between the two layers is a re- 
flective layer (14). The front and rear 
plates (8, 9) carry electrodes (10, 11) 
arranged in an xj matrix of separate 
addressable pixels. Projection means 
(1) project images onto the light sen- 
sitive layer (15) to vary the light re- 
flective properties of the liquid crystal 
layer (13). Thus when the liquid crys- 
tal layer (13) is scanned by the laser 
sources (6) in sequence the variable re- 
flectance across the layer provides an observer with a sequence of holographic images. 
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AUTOSTEREOSCOPIC DISPLAY 

This invention relates to the display of three dimensional (3D) video images on a 
video display. The term video is to be taken as the display of successive frames of 
5 still images collectively capable of providing a moving image. 

Most moving displays are two dimensional, for example television (video) and movie 
films. For many applications movement of two dimensional images is adequate 
because an observer can obtain stereoscopic information from movement. 

10 

In non-moving (still) displays two dimensional displays are most common and are 
quite cheap to produce, by e.g. photocopying, printing, photography. Two 
dimensional images are provided by displays whose amplitude (amount of darkness 
and or colour) varies with position within the image. True three dimensional still 
15 displays can be provided by holographic techniques with photographic plates for 
example. Other 3D effects can be provided by a technique known as autostereo. 

A three dimensional image can be provided in holographic films by variation of 
amplitude or phase within the hologram. An observer sees a slightly different image 
20 by each eye and at different positions (distance and angular) from the display. A big 
difference between the two types of display is the amount of information needed, with 
at least an order of magnitude more data needed for the 3-D case. 

Holographic images have been displayed using liquid crystal devices, but suffer from 
25 very narrow fields of view, i.e. the stereoscopic effect is only observed over a narrow 
range of angles. 

Moving video displays are formed by displaying frames of still images at a high 
enough rate that an observer can not detect individual frames, and instead sees a 
30 seamless movement. To display 3-D images with movement requires data rates 
much higher than is easily and cheaply available, especially for large displays. 
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According to one aspect of this invention the above problem of angle of view is 
solved by a three dimensional display which includes a spatial light modulator (SLM) 
on which a repeated sequence of patterns is illuminated by collimated light that scans 
5 repeatedly through a series of discrete directions in synchronisation with the 
holograms. 

The term "pattern" is used in this specification to mean a two dimensional variation in 
phase and or amplitude which could comprise the hologram, autostereo distribution 
10 or image to be projected. 

According to this invention a three dimensional video display includes: 

- a screen on which successive frames of data patterns can be displayed; 

- collimated light source that can be scanned repeatedly through a series of discrete 
15 directions in synchronism with the scanning of successive frames; 

- the arrangement being such that a three dimensional moving image can be 
observed by a viewer. 

The screen may be in the form of a liquid crystal light valve or optically addressed 
20 spatial light modulator (OASLM). A typical OASLM includes a layer of a light sensitive 
material operable to change the voltage on an adjacent layer of liquid crystal material 
and hence change its reflective properties when scanned by the collimated light 
source. For example the light sensitive material may be amorphous silicon. 
Alternatively the light sensitive layer may be a matrix of light sensitive diodes capable 
25 of transmitting an electric signal when illuminated. 

The OASLM may be formed by a front transparent plate carrying column strip like 
electrodes on its inside surface, next the liquid crystal layer, a reflective layer of e.g. 
aluminium arranged as small islands of separate dots, the amorphous silicon layer, 
30 and a rear transparent plate carrying row strip like electrodes on its inner surface. 

The photosensitive layer may be arranged for receiving video patterns. 
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In one embodiment the video patterns are imaged onto the light modulating layer as a 
succession of spatially separate patterns each forming a field with a plurality of fields 
collectively forming a frame of a display representing a single hologram. This 
5 technique of forming a large display from a plurality of smaller segments may be used 
in another embodiment to provide a large area projection display when illuminated 
with a single light source. It allows small fast addressed light modulators to form a 
composite display onto a slower acting light valve which is switched when a complete 
frame has been imaged. 

10 

The scanning may be provided by an array of laser sources which may be located at 
or adjacent the focal plane of a lens arranged optically in front of the image screen. 
The angle between each laser may be about the maximum angle which light from the 
hologram diverges. Increasing the number of lasers improves the angle of view in 
15 the observed display. 

To obtain a colour display, three different frequency lasers are used. 

The video images may be from a projector formed by a layer of ferro electric liquid 
20 crystal material on a silicon layer having an x,y array of individually addressable 
pixels and capable of very fast multiplex addressing. 

The projector may project its image through a grating formed e.g. by two orthogonally 
arranged linear gratings of alternate light transmisslve and opaque lines. 

25 

Alternatively the projector may be a video imager and direct its output direct onto the 
light sensitive layer. 

The video images to be displayed may be obtained in a conventional manner and 
30 stored for sequential display. For example a subject may be imaged by a plurality of 
cameras arranged at different angles in front of the subject to image the subject in a 
time sequential manner. 
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Ideally the images on a video display should be three dimensional like those 
recorded by holograms so that changes in parallax can be seen as the observer 
moves closer to or further from the display, and so that the observer can gauge 
5 depth by inspecting the image from different viewpoints and by using binocular ' 
vision. Holograms are merely high resolution displays and it is possible to display 
a video hologram on a liquid crystal display. However liquid crystal layers tend to 
be at least 1.5 microns thick, so the pixels on a liquid crystal tend to be no smaller 
than 2 or 3 microns, and it follows that the field of view of a hologram on a liquid 
10 crystal display is little more than a few degrees. Furthermore the cost of liquid 

crystal displays scales with resolution in such a way as to make the display of video 
holograms uneconomic. 

Three dimensional images can also be displayed using autostereoscopic pixellation, 
15 where the screen comprises a two dimensional array of pixels each of which rather 
than being Lambertian (as for the display of two dimensional images) controls the 
intensity of light as a function of ray direction. A simple way of making an 
autostereoscopic display is to illuminate a liquid crystal display with a continuously 
scanning spot source of light in the focal plane of a lens. The lens and light source 
20 produce rays all travelling in one general direction at any one instant, and if the 

direction of the rays is synchronised with the display of appropriate views of a solid 
object on the liquid crystal display, then the eye will integrate a three dimensional 
image over time. The problem here is that in order to avoid the observer seeing 
flicker, the liquid crystal display nnust switch quickly and for good quality three 
25 dimensional images this is once again uneconomic. 

Unless the positions of all observers are known then three dimensional images 
almost inevitably require data rates to be an order of magnitude greater than for two 
dimensional images. Any design of three dimensional display should provide a way 
30 of handling these data, but the higher the data rates, the less capacitance can be 
tolerated on the transmission lines which carry data onto the screen, and therefore 
the smaller the screen. 
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Small displays with high frame rates can indeed be made cheaply, for example by 
placing a layer of ferroelectric liquid crystal on a silicon substrate, or with a digital 
micromirror device or cathode ray tube. But users want large three dimensional 
5 images, and even with devices where a three dimensional image can be displayed, 
the effect of magnifying the image to a usable size is to make the field of view too 
narrow. 

Transmission line effects become less relevant if data is transferred in parallel, and 

10 this is what happens on light valves (or optically addressed liquid crystal displays) 
which comprise a sandwich of a light .sensitive layer and a light modulating layer 
such as liquid crystal. When a voltage is applied across the light sensitive layer and 
the light modulating layer together than an image projected onto the light sensitive 
layer is transferred across to the light modulating layer, and amorphous 

15 silicon/ferroelectric liquid crystal light valves have frame rates of several kilohertz. 
Furthermore the layers in a light valve are in principle unpatterned, so one can get a 
device which acts as a high resolution, high frame rate liquid crystal display without 
the cost of high resolution lithography. If images from a small display are projected 
onto the back of a light valve, it is possible to assemble a device capable of 

20 projecting high resolution two dimensional Images from relatively cheap 

components and various configurations have been proposed. The light valve can 
be larger than the liquid crystal display so can have higher resolution, but the liquid 
crystal display can nevertheless address all parts of the light valve if the image of 
the liquid crystal display is multiplexed across the light valve so that it addresses 

25 adjacent areas of the light valve one by one within each light valve frame. The 

magnification associated with projection makes these configurations inappropriate 
for the display of three dimensional images, but because they are unpatterned, light 
valves can in principle be made as large as television screens and still be relatively 
cheap. 

30 



wo 99/00993 



PCT/GB98/01866 



-6- 

Brief description of drawings: 

Tine invention will now be described by way of example only with reference to the 
5 accompanying drawings of which:- 

Figure 1 shows how one can display a 3D image by showing views of the object on a 
liquid crystal display and illuminating each to an appropriate direction; 

1 0 Figure 2 shows different volumetric 3D displays; 

Figure 3 shows that a 3D array of light emitters cannot display opaque images; 

Figure 4 shows three different holographic 3D displays; 

15 

Figure 5 shows different autostereoscopic 3D displays; 

Figure 6 shows. 3D object as seen through a pair of slits, one spinning rapidly, the 
other slowly; 

20 

Figure 7 shows that close to the screen a perspective image is seen whose 
composition can be determined by ray tracing; 

/ 

Figure 8 shows that a plot of pixel direction (9) versus pixel position (x) can be used 
25 to identify what the viewer sees on the display no matter how distant it is; 

Figure 9 shows a distant photograph of an autostereoscopic display on which each 
view comprises a horizontal bar; 

30 Figure 10 shows a close-up photograph of an autostereoscopic display on which 
each view comprises a horizontal bar; 



Figure 1 1 shows that an off-screen spot can be imaged by setting up rays to 
converge through it; 
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Figure 12 shows two views of a cubic array formed by parallel projection. The 
minimum angle (A0) between the views required for there to be a distinct difference in 
view content is that required to make one column of rear pixels fully visible; 

5 

Figure 13 shows two cameras, in the left hand video camera only the front pixels of 
the distorted cubic array are visible, while in the right hand camera one column of the 
rear pixels has become visible. As the depth of the distorted cubic array tends to 
infinity, RAY 1 and RAY 2 tend towards parallel, so the angle between adjacent views 
10 tends to the angle between adjacent pixels; 

Figure 14 shows a holographic display of the present invention. It shows how an 
autostereoscopic/holographic display with a wide field of view can be made by time- 
sequentially illuminating a high resolution liquid crystal display; 

15 

Figure 15 shows more detail of Figure 14, 
Figure 16 shows a simple 3D imager; and 



20 



Figure 17 shows a modification of Figure 14 arranged to show a large area two 
dimensional display. 
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Description of a simple 3D display as shown in Figure 16. An image screen 20 is 
caused to display a data pattern previously recorded. A scanning light source, e.g. 
an array 21 of laser emitters, illuminates a lens 22 behind the screen 20. The action 
of light scanning over the display provides a viewer with a 3D image. This shows that 
5 3D may be provided by projection, as well by reflection described below. 

Description of invention with reference to Figures 14 and 15. 

As shown in Figures 14 and 15 a display comprises in serial order, a projector 1, 
10 imaging lens 2, grating 3, light vaive 4, scanning lens 5. laser array 6. and an 
observer 7. 

For lower performance and cost the projector 1 may be one or more video 
projector{s) which project images onto the light valve 4. For higher performance, the 
15 projector 1 may be a ferro electric liquid crystal display capable of being multiplex 
addressed by a matrix of transistors to give a high speed image which is projected to 
the light valve. An example of this is a digital micromirror device (DMD) available 
from Texas Instrument Inc. 

20 The grating 3 may be formed by two linear gratings arranged orthogonally. Each 
grating has alternate light and opaque lines with a periodicity dependent upon the 
number of separate images to be projected onto the light valve 4. As shown the light 
valve 4 receives a 4x4 matrix of fields, to be described later. For this the wavelength 
of light divided by grating pitch is approximately equal to the angle between each of 

25 the four images on the light vaive. The mark space ratio of light to opaque areas of 
the grating is about 1 :3 or 4, 

The light valve 4 is a relatively standard item, such as an optically addressed spatial 
light modulator (OASLI\/l) sometimes called an optically addressed light valve; 
30 samples are available from Hughes Inc., Hamamatsu Japan, and Peterlab Ltd St 
Petersberg Russia. 



wo 99/00993 



PCT/GB98/01866 



-9- 

As shown the light valve is formed of a front plate 8 and a rear plate 9 both formed of 
glass. The size of the plates determines the size of observed display. For example 
the display may be 300 x 300mm or more. The front plate 8 carries four strip 
5 electrodes 10 formed as four column electrodes IO1.4. The rear plate carries four 
strip electrodes 1 1 formed as four row electrodes 1 1 1.4. Typically the gap between 
adjacent strip electrodes is about 10|.Lm, sufficient not to detectable by an observer 7. 
As shown the electrodes form a 4x4 matrix of segments or pixel (the term pixel is 
usually reserved for much smaller areas of displays), but could any other suitable 
10 number. For example a 1x2, 2x2, or 25x25 or more array as required. 

Between the plates 8, 9, separated by spacers 12 which may be distributed across 
the surface of the plates, e.g. pillars or spacer beads, is a layer 13 typically 1.5 to 
S^im thick of a ferro electric smectic liquid crystal material 13. Next is a light 
15 reflecting layer 14 of aluminium formed by numerous dots. Next is a layer 15 of 
amorphous silicon, sensitive to light imaged thereon. The wavelength of light 
projected onto the iayer 15 can be the same or different from that of the lasers 6.. 

Voltages are applied to the row and column electrodes by a control unit 16, and serve 
20 to change the physical arrangement of molecules within the liquid crystal layer 13. 
The unit 16 also times the loading of patterns, operation of the projector 1 , and 
switching of the lasers 6. 

The laser array of independently emitting lasers 6 is arranged in the focal plane of the 
25 scanning lens 5. Each laser 6 is arranged to illuminate the whole of the front plate 8. 

An observer 7 will see a holographic image on the front of the light valve 4 when a 
holographic pattern is illuminated by one of the lasers 6. Illumination by a sequence 
of the lasers 6 improves the angle of view, and a sequence of holographic images is 
30 observed as a moving holographic three dimensional display. 
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Operation to display a moving 3-D display will now be described: A series of pre- 
recorded holographic patterns may be provided e.g. from a computer store 17 or 
laser storage disc in a conventional manner. 

5 

A pattern is formed on the ferro electric projector display 1 and imaged through the 
lens 2 and grating 3 onto the area shown as "A" In the figure 14. At the same time a 
voltage is applied to row 1 and column 1 so that a voltage appears across the pixel 
shown as A. 

10 

The pattern projected is a finely detailed pattern containing phase information as part 
of a conventional holographic image. The light from this image appearing at "A" falls 
on the light sensitive layer 15 and alters its conductivity which in turn alters the 
voltage occurring within the pixel A. This provides minute variation of voltage and 
15 minute variation in physical arrangement of the liquid crystal molecules which gives 
rise to minute variation in phase information across the pixel. This is a first field. 

The above process is repeated for each of the pixels shown as "B" to "P" for fields 
two to sixteen. A total of sixteen field provides one frame or one complete 
20 holographic image within the liquid crystal layer 13. 

At the end of one frame all the liquid crystal material 13 has been arranged to 
represent a holographic image when illuminated by one of the lasers 6. 

25 A second frame is imaged onto the light valve 4 as in the first frame. This second 
frame is slightly different from the first frame and is illuminated by the second laser 6. 
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This is repeated for a third frame, and illuminated by the third laser 6, or continued for 
further frames if more than three lasers are used. For example, for a colour display a 
3x3 array of red green and blue light would be used, requiring nine separate frames. 
5 Use of more than one laser with associated frame improves the angle of stereoscopic 
view to the observer or observers 7. 

After three, or more frames, have been imaged and observed, a new series or three 
frames is imaged. A succession of different sets of three frames provides a moving 
10 three dimensional image to be seen by the observer 7. 

The display of Figures 14, 15 may be simplified to provide a large area two 
dimensional display as shown in Figure 17. This is similar to that of Figures 14. 15 
except that a single light source 6 illuminates the screen 4 and reflects light onto a 

15 large area display. Alternatively, the screen 4 itself may be made sufficiently large to 
provide the required size of display. This embodiment uses the technique of a small 
fast switched projector 1 (such as a DMD) forming a plurality of segment patterns or 
images to give a large size frame of information that can be switched at a slower rate 
(but still above an obsen/abie flicker rate) to observers. This technique has been 

20 termed tiling. When a 2D image is imaged onto the screen 4 a 2D image is seen by 
the observer. If a holographic pattern is imaged onto the screen 4, then a 3D image 
can be observed although with a reduced angle of view. Scanning successive 
patterns provides a moving 3D display as in Figures 14, 15. 



1 
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The text of the priority document is repeated below; the application then continues 
with the claims of this application. 

THE DISPLAY OF THREE DIMENSIONAL VIDEO IMAGES 

5 

I. introduction 

Conventionally televised images are two dimensional yet enable sufficient depth 

perception that surgeons, for example, are able to operate by them. Nevertheless 
10 when depth perception is critical, as it is in manipulative activities like surgery, depth 

perception is quicker and more reliable if the images have a three dimensional 

content^ Television and video games are likely to be more realistic with three 

dimensional images, and the interpretation of complicated data more simple, There 

has therefore been a renewed interest in three dimensional television and while there 
15 has been detailed work^ on the systems needed for this, a concentrated analysis is 

desirable of the component on whose radical evolution the rest of the system will 

depend: the display. 

It can come as a surprise to learn how little is needed to make a display for crude 
20 three 

dimensional images. For example one need merely take the liquid crystal display 
from 

a typical lap-top computer, swap the back illuminator for a lens and place a spot 
source of light some distance behind the lens, as show in: 

25 

Figure 1. The spot source might comprise a laser beam incident on a translucent 
screen, and under the action of the lens will illuminate the display with rays which 
converge to form an image of the source. Since the picture on the display will be 
visible only if observed from within the confines of the image of the source, the 
30 picture will have a restricted field of view, and is set up to be a view of a three 

dimensional object. Other views of the three dimensional object can be made visible 
to other areas by deflecting the laser beam to a different position for each view. If 
this operation is repeated at a rate sufficient to avoid flicker and if the whole of the 
plane of convergence is illuminated then the result will be a steady three dimensional 

I 
i 
I 

I 

i 
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image even, as will be shown later, if the display is viewed away from the plane of 
convergence. 

The three dimensional images formed by such a display will be crude because both 
the amorphous silicon transistors and the nematic liquid crystal typically found in a 
5 liquid crystal display switch too slowly to form many views, Furthermore the display 
will be small because the complexity of liquid crystal displays makes the manufacture 
of large devices expensive. 

This example conveniently illustrates the problems of three dimensional display. 
10 Because of the extra dimension, true three dimensional images require an order of 
magnitude more data than two dimensional images, and the first challenge faced by 
the designer is physically to distribute these data across the display's screen at a 
sufficient rate. Second is that of providing the screen itself with a sufficient space 
bandwidth product: i.e. enough pixels each switching sufficiently quickly to transfer 
1 5 the data into modulated light. Last is the challenge of enabling the manufacture of a 
display with these properties without requiring prohibitive precision or cleanliness. 

Each of these challenges is familiar to the designer of displays for high definition two 
dimensional images and it is arguable that, pixeliation and optics aside, three 
dimensional video images are merely a technological extension of their two 
dimensional predecessors, Nevertheless the variety of schemes recently put forward 
is bewildering, so this article will proceed by reviewing of some of the more 
successful technologies and showing that they comprise three distinct schemes of 
pixeliation, One of these, autostereoscopic, shows promise but is flawed at low 
resolution so the article goes on to calibrate this. The fourth section considers what 
resolution is needed for a flawless autostereoscopic three dimensional image, and 
shows that for a typical size of display one can do as well if not better with a 
hologram. The fifth section proposes a hybrid of autostereoscopic and holographic 
pixeliation which gives the advantages of both and the sixth section shows how 
photonic devices make the display of such images possible. The paper concludes by 
evaluating the bandwidth of the latest photonic devices, noting the trend in three 
dimensional display towards the integration of the display with the computer, and the 
future dependence of both on advances in gigahertz switches. 
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II. A review of three dimensional video displays 

A popular way of screening a three dimensional image is to extend the principle of 
conventional television to the third dimension by making a device capable of omitting 
5 light at any point in a volume (Figure 2). Perhaps the earliest way of doing this was 
to reflect light from a cathode ray tube off a circular mirror which vibrated like a 
drum^'"^. An image of the cathode ray tube formed at varying distances from the 
mirror thereby sweeping out a three dimensional volume, but the supporting structure 
was heavy and the field of view limited. Light emitting diode screens^'^ or laser- 
10 scanned displays^'^ have been used instead of a cathode ray tube, but the depth 
scanning mechanism remains cumbersome. 

An all but unrestricted field of view can be provided by spinning a two dimensional 
array of light emitters through a three dimensional volume. Among other ways this 
15 has been done with an array of light emitting diodes^, with a translucent screen which 
is scanned by lasers^^'^'"'^^ , and with a phosphor screen which is scanned (inside a 
vacuum) by electron beams^^ , The last has the advantage of a cheap screen and 
scanning mechanism, but any rotating screen system has a singularity at the axis of 
rotation. 

20 

An unrestricted field of view without any singularity can be provided by scanning a 
pair of laser beams across a transparent material which emits light where the laser 
beams intersect^"^'^^. An image of approximately one cubic centimetre has been 
demonstrated, but even a larger image would, like both vibrating mirror and spinning 
25 disc displays, provide only for the emission of light and not for its absorption. Each of 
these displays is therefore not able to provide opacity, so while the displayed images 
are three dimensional they are necessarily translucent (Figure 3). 

30 

Schemes for the display of opaque images have been proposed, for example 
stacking liquid crystal displays into a volume^^, but even if these were interleaved with 
light emitters the result would still be incapable of displaying reflections or specularity. 
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The advantage of a volumetric display is that it can provide an unrestricted field of 
view without excessive data-rates, requiring as it does a data-set of only three 
dimensions (the alternatives will be shown in effect to require four). This means that 
5 volumetric displays are not without potential application, for example in air traffic 
control or battle management. But the ideal is a device free of all optical restrictions, 
and one which with certainty can display any three dimensional image is the 
hologram. 

10 A hologram effectively freezes the optical wave-fronts scattered off a three 

dimensional object by recording their complex amplitude, and dynamic holograms 
are often proposed as a way of displaying a three dimensional image^ (Figure 4). 
A greyscale hologram is merely a high resolution two dimensional image, and 
conventional liquid crystal displays can be used to display such a hologram, albeit 

15 with a narrow (4° at present) field of view''®'''^. Wider fields of view require 

pixellation too fine for active matrix displays, but ferroelectric liquid crystal displays 

20 

can be made with high resolution at realistic yields . Nevertheless they still require 
a large number of connectors, and one scheme avoids this by scanning the back of 
an optically addressed liquid crystal display with a cathode ray tube^^'^^ . Even with 
20 this improvement the resolution of any liquid crystal display cannot be less than two 
or three times the cell gap, the result of which is to restrict the field of view of the 
hologram to a few degrees. 

Acousto-optic modulators provide phase modulation and have been used to display 
25 colour dynamic holograms^^'^"^ . The difficulties with scanning mirrors and bulk optics 
can to some extent be avoided^^, but once again there, is also the difficulty of 
modulating light at a resolution sufficient to get a wide field of view, which in the case 
of acousto-optic crystals with their high speed of sound would necessitate gigahertz 
acoustic waves the generation of which tends to melt the crystals. Even after this 
30 physical limitation data rates are necessarily high, whatever the pixellation of the 
image. They can be further limited for example by summing the hologram from a 
range of independently modulated spatial frequencies, although one is then not so 
much displaying a true hologram as displaying a three dimensional image 
holographically. 
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It is because of the need to reproduce optical pliase that the data rates of a true 
holographic display are extreme. But the human eye is no more sensitive to the 
phase of a three dimensional image than it is to the optical spectrum of a colour 
5 image, and just as colour images need compnse only red, green and blue primaries, 
a three dimensional image need comprise only the correct distribution of ray intensity 
versus position and direction that is specified by autostereoscopic pixellation. 

Autostereoscopic displays are named after their stereoscopic predecessors which 
10 require the user to wear spectacles. With the latest stereoscopic displays the 
spectacles comprise a pair of liquid crystal shutters which are synchronised to a 
screen that displays alternate left and right-eye pictures, and with a sufficiently high 
frame, rate the viewer sees a flicker-free image. The image provides stereopsis, i.e. 
the binocular perception of depth, but not kineopsis, which is the monocular 
1 5 perception of depth we accumulate by subconsciously moving our heads around a 
scene. Of the two, stereopsis is confined mainly to animals such as predators and 
primates who need to make instant estimates of depth, and it is arguable that even in 
these species kineopsis is a more relied-upon determinant of depth in static 
situations. Viewers can experience nausea after prolonged viewing of stereoscopic 
20 displays^^ and this may be due to subconscious awareness of the lack of kineopsis, 
but the real problem with stereoscopic displays is that spectacles get lost. 

Spectacles become unnecessary if each view is projected into one eye, and this can 
be done using the display described in the Introduction, This and displays like it are 
25 called autostereoscopic (Figure 5), a word which like television is an unhappy mix of 
greek and latin that seems peculiar to the display industry. One cannot expect a 
viewer to keep his or her head fixed merely for the convenience of the display, so one 
approach that is attracting great interest is to continually monitor the position of the 
viewers' heads and adjust the projection optics and view content accordingly. 

30 

More than one pair of eyes can in principle be tracked, and if the content of each, 
view is matched to eye position then the display can provide for both stereopsis and 
kineopsis. Furthermore it might be possible to display views with approximately the 
right perspective by guessing the distance of each viewer from the screen through 
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measurement of the distance between their eyes, so that almost the only missing 
depth cue would be accommodation (the ability to focus on off-screen pixels). While 
the resulting image would therefore be something short of truly three-dimensional, it 
is unlikely that viewers would notice. 

5 

The design of the display for such systems is relatively straight-forward because the 
data-rate of conventional video need be increased only by a factor of two (or four for 
two viewers etc)^^'^®, so that the major challenge becomes that of identifying and 
tracking the viewers. Demonstrators have been built which require the viewer to 
10 wear an infra-red reflecting spot^^ or a magnetic sensor^° but many authors 
are coy on their plans for tracking bare heads^^. 



An elegant approach is to side-illuminate the head with infra-red light so that one eye 
is illuminated and the other in shadow^^*^^ , but the shadows of more than one viewer 

15 can fail on each other. Another impressive approach is to track the hair/face 

boundary of viewers while a system which tracks the eye, nose and lips of a face 
has achieved 80% reliability with the face of the designer^^. But the latter is slow, 
tracks only one face, and is less effective with a variety of faces. Advances in 
technologies like speech, handwriting and object recognition mean that the day must 

20 sureiy come, when systems will be aware of their surroundings, but the development 
of such machine intelligence will herald a new generation of computing, and progress 
in these areas has so far been slow. Meanwhile the possibility of irritating glitches 
due to intermittently unfamiliar situations is never quite excluded, and users are 
notoriously intolerant of such weaknesses. 

25 

Multiple view autostereoscopy makes the position of the viewers' heads 
irrelevant because the display projects views to every position where a viewer might 
be. It will be left until the next section to convince the sceptical that such an image 
can be truly three dimensional, but with the need for a many-fold increase in 
30 bandwidth the, design of the display now becomes rather daunting. 
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The lenslet array is perhaps the longest established such autostereoscopic 
technology^®'^^ '*°''^\ first developed for three dimensional photography and now being 
applied to displays. Every lenslet covers a pixel from each view, and the lenslets 
combine to make the pixels comprising e.ach view visible to an exclusive direction. 
5 Although lenslets magnify the dead-zone between adjacent sub-pixels this can be 
smoothed out"*^, but the numerical aperture of simple lenslets restricts the field of 
view of lenslet displays to approximately 15°; outside this angle the three dimensional 
image repeats Itself which can be irritating. 

10 if an array of diffraction gratings is used instead of an array of lenslets. it is possible 
to get wider fields of view without dead-zones or repeating views'*^ but both 
grating and lenslet array displays require an underlying display whose resolution is 
the product of the view resolution and the number of views: a substantial 
manufacturing challenge. Nevertheless high resolution displays are in prospect, 

15 and the latest lenslet array displays assembled in laboratories have eight colour 
views at VGA resolution. 

High manufacturing yields are unnecessary if one makes a display by lining up 
several video projectors behind a lens^® "*^. In this system the projectors image one 
20 view each onto the lens, and the lens makes each view visible to a different direction. 
The projectors must be precisely aligned, must have uniform brightness, and the 
projection lenses must be carefully designed to adjoin one another without 
perceptible gaps. 

25 Both lenslet arrays and multi-projector systems multiplex the views of a three 
dimensional image from spatially distinct sub-pixels, but one can also use the 
persistence of human vision to multiplex video images over time. It is possible to 
take what amounts to a single lenslet with sub-pixels from a lenslet array display and 
raster scan it across a screen with spinning mirrors^® , but it is difficult to see how to 

30 multiplex across the screen in this way without moving parts. The alternative is to 
multiplex the views overtime, and witii the lenticular screen this can be done by 
replacing the lenticular screen with a low resolution array of siits'*^^°. Due to pin-hole 
optics the slits act at any instant like lenslets and with a low resolution display 
underneath produce a low-resolution three dimensional image. By scanning the slits 
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over the underlying display it is possible to time-multiplex the equivalent of a full 
resolution lenslet array but with no lens aberration and no need for high resolution 
sub-pixels. Slits however waste light and a less wasteful method of getting the same 
optical effect is to exchange the slit for line illumination^^'^^. Similar but perhaps less 
5 complex is the time-multiplexed concept described in the introduction^^"^'''^ . Both latter 
approaches have the great virtue of wasting no more light than a conventional liquid 
crystal display, but both require a liquid crystal display with a high frame rate. 

Poly-crystaliine silicon transistors and ferroelectric liquid crystals each switch an order 
10 of magnitude faster than their amorphous silicon and nematic predecessors, and a 
small liquid crystal display with a frame rate of I kHz has been demonstrated^^. 
Cadmium selenide and amorphous diamond transistors also switch quickly, and fast- 
switching grey scale modulation is possible with the distorted helix and electroclinic 
effects, with monostable or domain switching ferroelectric liquid crystals, and with 
15 anti-ferroelectric liquid crystals, However great resources were needed to develop 
even the existing liquid crystal displays, and greater confidence in the desirability of 
video three dimensional images will be needed before advanced liquid crystal 
displays are developed. 

20 A time multiplexed cousin of the multi-projector system can be constructed by 

replacing the several projectors with a single large projector whose projection lens 
covers the whole area filled by the multiple projectors'^ , and placing over the lens a 
mechanical^^'^®'^^ or liquid crystal^^ shutter which blocks light from all but one area. 
At any instant the projector does the same as one of the projectors in the spatially 

25 multiplexed system, but at successive instants different areas of the shutter are made 
transparent so that each view of the three dimensional image can be projected in 
turn. Careful alignment is unnecessary so a cathode ray tube can be used without the 
expense of beam indexing, Indeed the concept is so fault-tolerant that the author was 
able to assemble a crude, system from a cheap video display unit and a couple of 

30 fresnel lenses. 

In the contest between spatial and time-multiplexing it is the lenslet array which 
seems to be receiving the most attention from manufacturers, perhaps because the 
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raising of high-resolution yields is primarily a manufacturing challenge of a kind which 
they have faced so successfully in the past. Certainly the history of the 
semiconductor industry has been that resolution increases inexorably, but so has 
switching speed. The great success of the cathode ray tube lies in its ability to effect 
5 time-multipiexing, and the time-multiplexed projection system produced an image 
comprising eight VGA views several years ago. Despite being bulky and optically 
inefficient this system is robust and flexible and continues to use the high data rate of 
cathode ray tubes to produce image qualities in advance of lenticular arrays. It is 
perhaps ail the more remarkable that a crude concept with many similarities was built 
1 0 over fifty years ago by Baird^^'^^ . 

The latest autostereoscopic displays produce images in which each view is visible 
across an arc of 1°, and there is a consensus among those who have seen such 
images that they will suffice for initial applications. But if a display is to produce real 

15 three dimensional images then it should be able to project the image of pixels at 
various depths, and the viewer should see perspective which changes with their 
distance from the image. While it is clear that volumetric and holographic displays 
can do this, the description of autostereoscopic pixellation so far provided makes it 
less apparent that autostereoscopic displays can also project real three dimensional 

20 images. The next section aims to remedy this; 

Hi: Coarse autostereoscopic pixellation 

The first proposals for television conjectured systems of spinning slits, and it is 
25 instructive to consider what happens if a spinning slit is placed in front of the 

hologram of a three dimensional image. It is a common experiment to look at a three 
dimensional object through a spinning slit and it is seen that the scene is unchanged 
except for being dimmer and perhaps slightly blurred. A hologram should reproduce 
the wave-fronts of a monochromatic three dimensional image exactly, so a hologram 
30 seen through a spinning slit should also appear unchanged. What makes such an 
experiment significant is that the slit prevents superposition between light from areas 
of the hologram alternately exposed by the slit. So we can consider the hologram as 
an assemblage of independent slit-sized sub-holograms. The results of this 
experiment would be no different if a raster scanning hole were used instead of a 
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spinning slit, so a hologram can be further considered as a two dimensional array of 
hole-sized subholograms. 

The sub-holograrns are different from the pixels of a two-dimensional image in that 
5 the intensity of light is a function of direction from which the sub-hologram is 

observed, as well as a function of the sub-hologram's position. Since there are two 
coordinates of direction (azimuth and elevation) as well as two co-ordinates of 
position, a four co-ordinate system is required for the true reproduction of a three 
dimensional image. 

10 

Now imagine that a second spinning slit is placed some distance away from the first 
as shown in Figure 6, and that it spins sufficiently quickly that there is no Moire 
fringing between the two. We would expect the three dimensional scene to remain 
unchanged except for being yet dimmer and perhaps more blurred. Only light 

15 travelling from the first slit through the second will be exposed at a single instant, and 
if both slits are replaced by raster scanning holes of sufficiently small diameter then 
the light passing through both will necessarily constitute a single gaussian ray. 
Because the second hole exposes rays travelling to different directions alternately it 
destroys superposition between them. It follows that even if it uses entirely 

20 incoherent light, a system which modulates rays as a function of both position and 
direction will suffice to display a true three dimensional image, 

This thought experiment demonstrates that autostereoscopic displays have the 
potential to produce real three dimensional images, but the images will only be 
25 genuinely three dimensional if they are comprised of enough views, and the eight or 
so views available from existing autostereoscopic displays are too few. If the image 
is not then genuinely three dimensional, how different does it look? 

Taking the display described in the Introduction as our model of explanation, imagine 
30 that the illumination is collimated and that an eye is far from the liquid crystal display. 
,lf a single illuminator is switched on which shines light via the lens towards the eye, 
then the eye will see the whole of the view associated with that illuminator, if instead 
the eye is close to the screen, Figure 7 shows that the angle which the eye subtends 
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to one side of the screen will be, different from the angle which it subtends to the 
other side of the screen, so the eye will see only part of the view. 

The picture integrated by the eye over time can be determined by following rays 
5 backwards from the eye pupil through the display to their source. Tracing rays 
backwards, a virtual image of the eye pupil will be formed by the lens somewhere 
behind the illuminators. If rays are now traced forward from this virtual image 
through the boundary between each illuminator, the rays will outline zones on the 
liquid crystal display. Each zone will be illuminated only when the illuminator outlined 
10 behind it is switched on, and since the liquid crystal display shows a different view for 
each illuminator, adjacent zones will show slices of different views. The result is a 
picture seen by each eye which comprises a mosaic of sections from each view, and 
if the view spacing is too coarse then there will be flaw lines between the view 
sections. 

15 

It is the coarseness of view spacing which causes the flaw lines rather than 
autostereoscopy itself, because the process described above is exactly how one 
gets perspective with a real object. Far from the object an eye will see a view 
comprising a parallel projection of the object in that direction, but close up the eye 

20 will subtend a different angle to one side of the object than to the other. So the eye 
will see rays from one side of the, object that are part of a different parallel 
projection from rays from the other. Figure 8 shows this schematically, and rather 
conveniently it proves possible to synthesise the diagram of Figure 8 on an 
autostereoscopic display by configuring each view to comprise a horizontal bar: the 

25 left-most view with the bar at the top of the screen, the right-most with the bar at the 
bottom, and the remainder spaced evenly between. The result is an optical test 
which calibrates the accuracy of the three dimensional display, and photographs of 
an 8 view-sequential projection display are shown in Figure 9 and Figure 10. 

30 It is tempting to suppose that the 3D image on a shuttered cathode ray tube would 
be smoother if the scanning shutter was scanned continuously as each view was 
written on the cathode ray tube instead of being moved by a single shutter width 
between the display of each view^^ . The idea is that this might smooth 
discontinuities between adjacent views which arise when the angles between them 
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are too coarse. Considering only the horizontal dimension, assume that as the 
cathode ray tube traces out the x coordinate, the shutter gradually moves by one 
shutter width. This gives a gradual change, of 9 with x so the pixellation in the x/G 
diagram is slanted. There will be a distance from the display where someone 
5 looking at it will see a picture that can be represented on the diagram by a diagonal 
which is parallel to the pixellation lines, so that at this distance the viewer will we a 
single view. This is exactly the result we would get if we put a weak lens in front of 
the screen of a conventional autostereoscopic display, so the 3D image is not 
smoothed but merely distorted. 

10 

Tolerant as the eye is of flaw lines between views, they nevertheless remain 
apparent. The claim that an autostereoscopic display produces a true three 
dimensional image can only be valid if the spacing between views is sufficiently fine, 
but just how fine is sufficient? 

15 

IV: 3D Pixellation 

The spacing of 1° per view that was reported in section II to be satisfactory for the 
present generation of displays requires 60 views for a typical field of view of 60°. It is 

20 tempting to state that flaws wilt only be eliminated on an autostereoscopic display if 
views are as finely separated as the human eye can resolve®'^ , but it was one of the 
breakthroughs in the development of two dimensional video to realise that such rigour 
is unnecessary. . This section assumes that a three dimensional image will be 
acceptable if with the same pixel dimensions as the equivalent two dimensional 

25 image it can be displayed without flaws. 

The volumetric array is the format in which computer aided design images are usually 
stored (indeed perhaps this is how our minds memorise three dimensional images) in 
which case the angle between each view need be no finer than the minimum 
30 difference in projection angle needed to render two views of such an array distinct. 

Holograms are physically perfect three dimensional images so have no flaws 
between views, and in weak moments proponents sometimes claim that ail else is 
mere compromise. But as the previous section demonstrated a hologram is to the 
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eye just an autostereoscopic display where view direction is controlled by diffraction, 
so holograms will also subtend a measurable angle between views which although 
too fine to see, will be finite, There will therefore also be a calculable depth of field, 
even for a hologram. 

5 

It is by relating the depth of field and angle per view between each pixellation scheme 
that the resolutions of differently pixellated images can be matched, and since one so 
often needs to display images of one format on a display of another, this section aims 
to formulate these relationships. 

10 

Dealing first with a cubic volumetric array, geometric optics is sufficient to determine 
the angle through which a video camera must move before its image of the array is 
substantially changed. Starting the camera far from the, array but with sufficient 
magnification that each pixel at the front of the array maps onto one in the video 
15 camera, there will be a certain sideways distance through which the camera must 

move before the column of pixels at one side of the rear of the array map onto a fresh 
column of video camera pixels. 

Figure 12 shows that the angle (A0) subtended by this distance to the centre front of 
20 the array equals the width of one array pixel (Ax) divided by the depth (z) of the array. 



If the width of each pixel equals its depth and the array is n, pixels deep it follows 
25 that: 



So the effective angle between views of a volumetric display is the reciprocal of the 
nunriber of depth pixels, and the, angle subtended by each view on an 
30 autostereoscopic display must equal this if it is to show a flawless image of 

equivalent depth. This means for example that the 3D equivalent of a VGA image 
comprising 640 by 480 pixels will need approximately 480 views in azimuth if it is to 




(1) 



A 0 = 



(2) 
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represent an array as deep as it is high over a field of view of 60^^ (equal to about 
one radian). 

Volumetric displays can usually image only finite depths, but in principle 
5 autostereoscopic and holographic displays can act as windows into a three 
dimensional environment. If the environment is infinitely deep, comprising for 
example an object with mountains in the background, must the angle between views 
be infinitesimal? 

10 The mistaken assumption in this question is that views of such an environment will be 
. formed by parallel projection i.e. to assume that views are formed by imaginary 
cameras far from the scene. In reality the projections the cameras form will be not 
parallel but perspective, and each camera will be able to resolve fine resolution close 
to, but coarse resolution far away. The smallest object which can be resolved at any 

15 distance from the camera is equal to the width of view visible at that distance divided 
by the number of pixels per line in the camera, Rather than a uniform cubic array a 
more appropriate test image is an array of voxels (volumetric pixels) in which the 
voxel dimension is proportional to the distance of the voxel from the camera: i.e. a 
distorted cubic array (Figure 13). 

20 

Through what angle can the direction of projection be rotated before the projected 
image changes? If one rotates about the frontal centre of the cubic array, the limit on 
rotation without change is set once again by the rear voxels of the array. There will 
have been an unambiguous change in the projected image once the direction of 
25 projection has been changed sufficient to translate the image of the rear voxels by 
one voxel diameter. Simple geometry shows that as the depth of the cubic array * 
tends to infinity, this angle equals the angle subtended by two voxels at the rear of 
the array to the camera. 

30 If each pixel in the image plane of the central camera is mapped to a voxel at the rear 
of the distorted cubic array, then the angle through which the camera can be rotated 
before the image changes equals the angle between its aperture and two adjacent 
pixels in its image plane. It follows that in order to televise a pixellated three 
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dimensional image of an infinitely deep scene the angle (A0) between adjacent 
cameras sfiould equal the cameras' field of view (a) divided by their resolution (nx). 

Ae=— (3) 

5 

If such an image is to be accurately reproduced on an autostereoscopic display then 
the angle at which rays from the edges of the display's screen converge should equal 
the field of view of the cameras. This is so that if the display is substituted for the 
original scene, the image recorded by the cameras is unchanged. Equation (3) 
10 therefore sets the angle between views on the display, so if a display with a field of 
view of 60° has VGA resolution views and is to act as an infinitely deep 3D window, it 
needs approximately 640 views in azimuth. 

While these translations between volumetric and autostereoscopic pixellation are 
15 correct geometrically, the angle per view of an autostereoscopic display is limited, 
and that of a holographic display determined by the laws of diffraction. 

The angle between views (AG) on an autostereoscopic display cannot be less than 
the angular divergence (56) of the rays which constitute each view, which is 
20 determined by the wavelength {X) and the pixel diameter (Ax) according to the. law of 
diffraction^^: 

59 = — (4) 
Ax 

so: 

25 AQ^— (5) 

Ax 

Continuing with the example of a display acting as an infinitely deep 3D window with 
a field of view of 60° and VGA resolution views, the angle between views according 
to equation (3) was 1/640 radians. So with red light (?.=633 nm) equation (5) 
30 stipulates that the pixel size cannot be less than 0.4 mm. This is the approximate 
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size of a pixel on a typical VGA monitor, so the restriction placed by diffraction on 
flawless autostereoscopically pixellated images is remarkably tight. 

The depth of field (z) of an autostereoscopic display is the maximum distance above 
5 the screen at which light can be made to converge (as shown in Figure 11) to form 
the image of a pixel of diameter Ax. By trigonometry this distance approximately 
equals the pixel diameter divided by the angle of ray divergence: 



10 



15 



<- (6) 
59 



Equation (6) is essentially the same rule of geometry as equation (1) but referred to 
the co-ordinates of the display rather than those of the camera. Combining this with 
the law of diffraction given by equation (4) gives: 

z<^ (7) 



In section III it was noted that although a three dimensional image can be seen 
through a pair of raster-scanning holes, the image will be slightly blurred, It is 
diffraction which causes this blurring, and if the diameter of both holes is Ax then 
equation (7) sets the maximum distance z between the scanning holes. Should the 
20 distance nevertheless be made greater than this then diffraction through the second 
hole would filter detail rastered by the first scanning hole such that its effective size 
would increase to that allowed by equation (7). 



25 

Section ill also noted that one can represent a single dimension of autostereoscopic 
pixeilation by plotting 6 versus x. But it is well-known that the spatial frequency k of 
the intersection between a plane and light waves intersecting the plane at angle 0 is 
given by 

30 k= —sine (8) 
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If autostereoscopic pixeDation is represented instead by a plot of k versus x then the 
dimensions of pixeilation are limited by combining equations (8) and (5) to get the 
classic expression: 



Depicting autostereoscopic pixeilation as an array of independent sub-holograms of 
diameter Ax, equation (9) reaffirms that the minimum increment in spatial frequencies 
10 Ak which can be resolved by each sub-hologram is equal to 2n divided by its width 
AX. 

Consider the effect of equation (7) on the three dimensional equivalent of a high 
resolution monitor, where pixeilation can be as small as 90 ^m^^ . The maximum 
15 depth of a cubic array would then be only 16 mm, so autostereoscopic systems are 
fundamentally inadequate for high resolution 3D images for which only holographic 
pixeilation will suffice. 

The angle between views on a hologram is also governed by the law of diffraction but 
20 because of coherence the maximum width of any ray equals the width (x) of the 
hologram, so that: 



But for a hologram the ray divergence does not equal the minimum angle between 
25 views because in the far-field a single ray does not comprise a view. Instead if a lens 
in the far-field is to form a view n^ pixels wide, then by conservation of information the 
lens must image a ray bundle nx pixels wide (or nx/2 if the modulation of the rays is 
complex). The minimum angle of view of a hologram is therefore nx times the 
minimum ray divergence: 



5 



Ak.Ax>2n 



(9) 




(10) 



30 



A e > 



(11) 



X 
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The iens will have to be moved through this entire angle before it forms a new and 
independent view, and if it is merely moved part-way between then it will form a 
superposition of the adjacent views. 

5 The choice of nx in this instance is somewhat arbitrary, but once made then the 
smallest area (Ax) which can be resolved on the, hologram is by simple geometry: 

Ax = ^ (12) 

X 

Combining this with equation (11) gives: 

10 

A0 > — (13) 
A A- 

This is exactly the same as equation (5), so if a hologram's pixel size is defined to be 
15 , the same as that of a diffraction limited autostereoscopic image both have the same 
angle per view, and therefore both have the same information content for the same 
quality of three dimensional image. Since it has already been shown that a flawless 
autostereoscopic image with equivalent resolution and size to a conventional VGA 
monitor Is at the diffraction limit, it follows that under typical conditions a flawless 
20 autostereoscopic image contains no less information than a hologram. 

Diffraction effects will not matter with large displays, nor when cameras are imaging 
large scenes. But if high resolution images are being formed of small-scale 
phenomena as would be required for example in 3D keyhole surgery, diffraction 
25 effects in the 3D camera will need to be considered and will obey rules similar to 
those given above. 

The depth of field of a hologram is found by combining equations (6) and (10): 



30 



(14) 
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So a hologram of width x=20 cm for example, illuminated by light of wavelength 
?i=500 nm, could in theory project a spot of diameter Ax= 100 |.im up to 40 m from its 
surface. Closer to its surface the smallest spot which a hologram can project is equal 
to its resolution, which in principle can be as small as one wavelength of light. Table I 
5 summarises these relationships. 

It has been shown that even though a hologram has the same information content as 
a typical flawless autostereoscopic image, it can resolve much smaller pixels to much 
greater depths. But the central problem with holograms - that wide fields of view 
10 require sub-micron spatial periodicity - remains. Is there no intermediary which 
shares some of the resolution and depth of holographic pixellation with the relaxed 
spatial periodicities permissible with autostereoscopic pixellation? 

One class of autostereoscopic displays is inherently excepted from the restrictions 
15 described in this section - those which use bulk optics. These are a special case 

because light is coherent across the optical wave-front, so although if all the screen is 
opaque except for one pixel then light will diffract as with incoherent autostereoscopy, 
if several adjacent pixels on a bulk optical display are transparent then light will 
diffract less. Indeed one can imagine writing a zone plate on such a display in order 
20 to cause an off-screen pixel to come into focus somewhere above the screen. 

Displays of this kind make possible an intermediary between autostereoscopic and 
holographic pixellation by combining them. 

25 

V: Hybrid pixellation 

It is proposed to combine autostereoscopic and holographic pixellation into a hybrid 
scheme which instead of projecting a series of views to different directions, projects a 
30 series of holograms to different directions. For example the concept described in the 
Introduction could be adapted to the new scheme by reconfiguring the liquid crystal 
display as a hologram, and reconfiguring the illumination to scan through a series of 
discrete angles (instead of continuously as before). 
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It is no longer adequate to use geometric optics to render a volumetric data-set into 
the sequence of patterns to appear on the liquid crystal display, and one is instead 
obliged to revert to the more rigorous regime of Fourier optics. 



The complex amplitude E(k) of light emitted in any direction to a distance far from the 
object is proportional to the complex sum from all parts of the object of rays in that 
direction. This far-field distribution should be determined with the necessary angular 
resolution, approximately equal to the wavelength of light divided by the diameter of 
10 the object. It is well-known that the near-field distribution of light is then the spatial 
Fourier Transform of the far-field distribution^^. 



where K is conventionally infinite but in this case need equal no more than the field of 
15 view divided by the wavelength. The hybrid scheme requires that this near field 
distribution be separated into a series of n holograms each to be projected to a 
different section of the far field. If the far field is divided into n sections, each 
subtending an angle Xx, then 



(15) 



E{k)=Yu - qY:)-h{k-{q + l-)ic (16) 



20 



25 



where h represents the unit step function. It follows that 



E{x)=^ ^[hik - qK) - h{k -iq + ])K]E{k)c^^lk (17) 



(18) 



7 = 1 qK 



wo 99/00993 



PCT/GB98/01866 



- 32- 



(19) 



(20) 



it 



(2.1) 



10 



15 



20 



25 



Where FT denotes the operation of taking a Fourier Transform, and Eq(x) denotes the 
Fourier Transfornfi of E(k+qK) frequency shifted by qjc. A lens focuses parallel rays of 
light so as to produce in its focal plane the distribution of complex amplitude which in 
the absence of the lens would be the far-field. The lens is therefore said to execute 
an 

optical Fourier transform on light travelling from one Fourier plane to the other. A 
spot source of light constitutes the function 5(x), and if this is placed in the focal 
plane of a lens then parallel wavefronts vyill appear in the far focal plane with 
distribution FT[5(k-qz)]. A suitable liquid crystal display will modulate these 
wavefronts with the function Eq(x), and time sequential addressing will sum the 
series, albeit as a summation of far-field intensifies rather than a sum of complex 



amplitudes as specified by the algebra. That the result is nevertheless a three 
dimensional image Is because, as ever, the eye is insensitive to phase. 



The field of view of a hologram Is by the law of diffraction approximately equal to the 
wavelength of light divided by the pixel diameter, so the field of view (0) of a hybrid 
three dimensional image is related to the wavelength (A.), number of views (no) and 
the pixel diameter (Ax) by: 




(22) 
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Assuming a flicker rate of 50 Hz, the frame rate of the liquid crystal display must 
equal 50 rio Hz and the spatial periodicity must equal the reciprocal of Ax, so the 
space-time periodicity must equal fifty times the field of view divided by the 
5 wavelength. For a one radian field of view in azimuth with a wavelength of 500 nm, 
the space-time periodicity should approximately equal I Mbit s'^ cm"^: well above 
the 5 kbit s"^ cm"^ capabilities of large high resolution liquid crystal displays^^. 



Hybridising the autostereoscopic and holographic pixellation schemes requires that 
10 there be coherence between adjacent pixels, and this is a fundamental property of 
displays with a single light source and bulk optics. In principle therefore the 
shuttered cathode ray tube system can also display hybrid pixellation images and it 
has already been noted that the system can screen data at higher rates than that 
based on liquid crystal displays. 

15 

In the description of coherent optics the two lenses in the shuttered cathode ray 
tube system can both be considered as effecting optical Fourier transforms^°. 
Adopting a classical layout, the cathode ray tube, should be in the rear focal plane 
of the rear lens, and the liquid crystal shutter in the focal plane shared by rear and 
20 front. 

It is well known that if one takes the Fourier transform of a function twice over the 
original function results, albeit with a change of sign. The same is true with optical 
Fourier transforms, so in the front focal plane of the front lens there appears an up- 
25 side-down image of the cathode ray tube screen. It is by spatially filtering the 

optical Fourier transform of this image that the slit in the focal plane shared by both 
lenses makes each view of the cathode ray tube image visible to a different 
direction. 



30 This set-up, however, cannot effect the hybrid pixellation scheme, not so much 

because individual pixels of the cathode ray tube do not cohere (the device could, 
after all, be swapped for something else) but because if the slit were narrowed to a 
spatial impulse it would filter out all the low frequency detail from the source. One 
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could instead have impulses in the spatial plane and a liquid crystal hologram in the 
Fourier plane, but this has few advantages to offer over the single lens approach. 

It might seem premature to be considering the ultimate resolution of three 
5 dimensional video images when present resolutions are so much lower. But while 
the resolution of spatially multiplexed and time multiplexed autostereoscopic displays 
is limited by spatial resolution and frame rate respectively, what the hybrid approach 
offers is the ability to interchange spatial and temporal periodicity. It is then the 
product of spatial and temporal periodicity which determines what three dimensional 
10 resolution a device makes possible, and devices already exist with the space-time 
periodicities necessary for high resolution three dimensional images. 

VI: Advanced 3D displays 

15 If liquid crystal displays lack the space-time periodicity needed for hybrid pixellation, 
one device stands out for its lack of complexity and high space-time periodicity: this is 
the light valve (also known as an optically addressable spatial light modulator). 
Section II notes that video holograms have already been screened by optically 
addressing such a device with a cathode ray tube but the field of view was narrow. 

20 With time multiplexed illumination it is possible to get a wider field of view and the 
frame rate of the latest light valves makes this possible^® . Figure 14 shows how the 
light valve can be addressed by multiplexing the image of an array across its rear, 
and high frame rate arrays now have the space bandwidth product needed to 
address light valves over large areas. The great advantage of this approach is that it 

25 removes from the screen the two most expensive items: the active matrix transistors 
and the connector array. One is then left with a screen which may be large but is 
uncomplicated and a small video projector which may be complicated but is not large. 
Both devices are therefore potentially cheap, and it is encouraging to note that 
arguably it Is exactly the same division of size and complexity between phosphor 

30 screen and electron gun which made it so economic to manufacture cathode ray 
tubes. 

While hybrid pixellation provides for flawless three dimensional images it remains 
unclear that users object to minor flaws, and autostereoscopic pixellation would 
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certainly be the simplest to implement on such a device if it were fast enough. But the 
frame rate of the latest light valves seems to be limited to approximately 2 kHz by the 
RC time constant of the amorphous silicon. Dividing this by 3 for colour and by 60 for 
flicker one might get 30 views, but if these views have a typical 640 pixels per line 
5 and are taken by cameras with a view of half a radian (approximately 30°) then 
according to equation (2) for a flawless image the angle per view should be 1/1280 
radians and the field of view of the device would be less than 1/40 radians 
(approximately 1.5°). Of course 30 views at the 1° per view which seems acceptable 
for the first generation of video three dimensional images would result in a 
10 satisfactory field of view. But the optically addressed system is not a flat paneL and 
with autostereoscopic pixellation would produce a three dimensional image little 
better than the flat panel active matrix liquid crystal display. The extra cost of the 
latter will eventually depend on how many got made, but in large quantities might be 
low enough to win over optical addressing. 

15 

A typical light valve can resolve down to 10 microns, which with a frame rate of 2 kHz 
gives a space-time periodicity of 2 x 10"^^ m'^s"\ After dividing by 3 for colour and 60 
for flicker, one can estimate the solid angle available for viewing by multiplying by }} , 
equal approximately to (0.5 x 10'V The result is a solid angle of view of 0.025, 
20 equivalent to a viewing zone of say 30° in azimuth by 3° in elevation. 

Light valves are likely to be able to do better than this. Frame rates of 5 kHz have 
• been reported®^ at the penalty of intense illumination (and a bistable liquid crystal), as 
have spatial resolutions of 5 microns. But before drawing optimistic conclusions one 
25 should consider the problem of writing data to these devices at rates approaching 
400 GHz for a 16 cm by 12 cm screen. 

An optical fibre is capable of transmitting data at such rates, and a simple method of 
scanning its output would be a tremendous prize both for displays and 
30 telecommunications. But existing acousto-optic devices can barely scan at I MHz, 
and optical amplifier arrays remain rather elementary. It was research into photonics 
which led to fast-switching light valves, and it is research into photonics which is 
producing some of the more promising ways of addressing them. If the addressing 
problem is simplified by requiring an image which is three dimensional only in 
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azimuth, then for a 16 cm wide screen with 240 interlaced lines the data rate reduces 
to (frame rate x 1/lateral resolution x width x no. of lines) = (2000 x 10^ x 0.16 x 
240/2) = 4 GHz. This brings the data rate within the range of existing devices and 
five stand out: acousto-optic holograms, cathode ray tubes, laser diode arrays, 
5 ferroelectric arrays and micro-mirror arrays. 

Acousto-optic holograms have a successful history but are limited by the speed of 
sound in acousto-optic materials which at 2 km/s restricts data rates to approximately 
4 GHz for an optical wavelength of half a micron. In practice even these rates are 
10 difficult because of the melting problems mentioned in section II. 

Cathode ray tubes can be electrostatically scanned at megaheri:z line rates and 
providing the deflection angle is narrow and beam intensity not too high the spot size 
can be kept to a diameter of a few microns. But it is difficult to make such a small 
15 spot bright without defocusing, and a way must be found of modulating an electron 
gun at more than 1 GHz, so while these challenges are not insuperable they remain 
challenges. 

Laser diode arrays and the other arrays work by de-multiplexing the input to a 
20 sufficient resolution that raster scanning is either not required or need be no faster 
than can be executed by a liquid crystal hologram. An 18 x 1 laser diode array has 
been operated at 8 x 1 GHz, and 256 x 256 arrays have been fabricated offering the 
tantalising prospect of space bandwidth products far in excess of any alternative. 

25 Ferroelectric arrays are fast-switching liquid crystal displays where the active matrix 
transistors are etched in a silicon integrated circuit. A 320 x 240 array with a potential 
frame rate of 20 kHz has been demonstrated^^, offering a space bandwidth product 
of 3.2 GHz. This is just about enough for a 16 cm wide screen, and it is 
demultiplexed enough for the array's image to be scanned across the light valve at 

30 the 20 kHz rates achievable by a one dimensional ferroelectric liquid crystal 
. hologram. Deflection angles would be small (0,5 mm divided by say 20 cm from 
hologram to screen equalling 1/40 radian) so the resolution of the one dimensional 
hologram would be a wavelength divided by 1/40 radian, approximately equal to 20 
microns. 
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Micro-mirror array liave the advantage of being comprised entirely of silicon 
although they require a more intricate lithography. Nevertheless arrays of 2048 x 
1 1 52 pixels potentially offer a space bandwidth product of 5.8 GHz. Details of 
5 circuitry aside this offers the potential for a screen over 8 inches wide, and if three, 
such devices were operated in parallel (which is how they are configured for high 
definition 2D projection) then one could hope for better quality still, But once again 
optimistic conclusions are inappropriate, in this case because these devices merely 
convert data from an electronic form to an optical one: there is still required a source 
10 of data. 

Whatever the capabilities of optical fibre, it seems highly probably that three, 
dimensional images will be compressed. The convention at present is that displays 
are connected to the video source by a cable and that any decompression is effected 

15 by the video source. But the data rates of raw three dimensional video are so high 
that it seems pointless to decompress the signal remote from the display, merely then 
to be presented with the challenge of transmitting a raw signal. - Rather the 
decompression should take place as close to the addressing device as possible 
(perhaps even within the addressing device) and it is convenient that both micro- 

20 mirror and ferroelectric arrays are mounted on carriers which plug directly into a 
printed circuit board. The complexity and output data rate of existing interfaces for 
three dimensional video suggests that the decompression machine will have 
computational power comparable to that of a typical computer, and with the current 
trend for the display to dominate the cost of a computing system it must be 

25 questioned whether there continues to be any advantage in going to the effort of 
separating the computer from the display. 

This section has brought the paper to a conclusion by attempting to demonstrate in 
some detail that it is practicable with existing technology to display a medium-sized 
30 colour three dimensional video image with no moving parts, an adequate field of view 
and no flaw lines. Three dimensional video is not some remote or esoteric prospect: 
it is a viable, analytic technology and its development, like that for two dimensional 
video, will depend on further progress in the three fundamentals of display technology 
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- spatial demultiplexing, screen space bandwidth product, and low cost per unit 
screen area. 
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VII: Conclusions 

Video three dimensional images can be pixellated in tliree ways: volumetric, 
5 holographic and autostereoscopic. Whiie volumetric images use Bandwidth efficiently 
to give all round viewing and holographic displays have high resolution, 
autostereoscopic displays image opaque objects with the wide fields of view needed 
for most applications. 

10 Autostereoscopic displays which track viewers' heads offer the prospect of greatly 
reduced data rates, but multiple-view autostereoscopy avoids the need, for machine 
intelligence and the latest such displays time-multiplex views to get the 1° view 
spacing which seems adequate for the first generation of three dimensional displays. 

15 Although acceptable in the short term, the images on autostereoscopic displays with 
l^per view are flawed and may come to irritate. For true three dimensional images 
the angle per view must be approximately 1/10° for an image 640 pixels wide, At this 
spacing even an autostereoscopic display with pixel diameter as big as 0.5 mm will 
be diffraction limited, and its data content no less than that of a hologram. 

20 Holograms have greater depths of field than autostereoscopic images, much greater 
resolution, and are virtually the only option for pixel sizes finer than 0.5 mm. 

Holographic and time-multiplexed autostereoscopic pixellation schemes can be 
combined to give a hybrid which has the virtues of both. A sequentially illuminated 
25 holographic display has the same data content, resolution and depth as a hologram, 
but the field of view of an autostereoscopic display. In principle all that is needed is a 
liquid crystal display with a space-time periodicity of the order of 1 Mbit s'^ cm', but 
this is impractical over large areas at low cost, 

30 Faced with the demand for high space-bandwidth products the optical 

communications industry has developed light valves and high frame-rate arrays 
sufficient to get the requisite space-time periodicities over large areas. Light valves 
are simple enough to operate over screen-sized areas at low device cost and the 
arrays provide a way of spatially distributing data across the light valve which need 
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not be expensive provided they are small.- Projecting a small array onto a large light 
valve therefore gives a display which is cheap and has high resolution for the same 
reasons that the cathode ray tube does. 

5 The high frame rate array should be as close as possible to the electronics which 
decompress the three dimensional image in order to minimise high data rate 
connections. The computational power of the decompression electronics will be 
comparable to that of most computers, and it is not unlikely that the computer and 
display systems will therefore come to merge, 

10 

The progress of two dimensional video has since its invention been one of steady 
evolution towards increasing resolution and size, drawing on parallel advances in 
telecommunications. While the display of video three dimensional images may seem 
revolutionary, this paper has sought to show that the pixellation and display optics are 

15 not unduly sophisticated and that the remaining challenges are the same as for two 
dimensional video: an increase in the screen's space bandwidth product, an increase 
in the rate at which data can be physically distributed across the screen, and the 
attainment of both in a single system without great complexity of manufacture. 
Photonic components developed for optical telecommunications already meet the 

20 requirements for three dimensional video, and the two technologies are likely to 
continue to interact to their mutual benefit. 



25 



30 
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Figure 1: One can display a three dimensional image by showing views of the 
object on a liquid crystal display and illuminating each to an appropriate direction 

Figure 2: Volumetric 3D displays 

5 Figure 3: A three dimensional array of light emitters cannot display opaque 
images 

Figure 4: Holographic 3D displays 

Figure 5: Autostereoscopic 3D displays 

10 Figure 6: A three dimensional object looks the same when seen through a pair 
of slits, one spinning rapidly, the other. slowly. 

Figure 7: Close to the screen a perspective image is seen whose composition 
can be determined by ray tracing. 

15 

Figure 8: A plot of pixel direction (9) versus pixel position (x) can be used to 
identify what the viewer sees on the display no matter how distant it is. 

Figure 9: Distant photograph of an autostereoscopic display on which each view 
20 comprises a horizontal bar. Permission for Reprint, courtesy Society for Information 
Display, 

Figure 10: Close-up photograph of an autostereoscopic display on which each 
view comprises a horizontal bar. Permission for Reprint, courtesy Society for 
25 Information Display. 

Figure 11: An off-screen spot can be imaged by setting up rays to converge 
through it. 

30 Figure 12; Two views of a cubic array are formed by parallel projection. The 

minimum angle (A9) between the views required for there to be a distinct difference in 
view content is that required to make one column of rear pixels fully visible, 
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Figure 13. In the left hand video camera only the front pixels of the distorted cubic 
array are visible, while in the right hand camera one column of the rear pixels has 
become visible. As the depth of the distorted cubic array tends to infinity, RAY 1 and 
RAY 2 tend towards parallel, so the angle between adjacent views tends to the angle 
5 between adjacent pixels. 

Figure 14: An autostereoscopic/holographic display with a wide field of view can 
be made by time-sequentially illuminating a high resolution liquid crystal display. A 
high resolution liquid crystal display can be assembled from a light valve and a high 
10 frame rate array. 

Table 1 : The angle per view (A9) and depth (z) of the three pixellation 

schemes can be related by the width of the image (x), the depth of the image in 
pixels (n^), the width of the image in pixels (nx). the field of view of the image (a), the 
15 pixel size (Ax), and the wavelength (A.). 
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TABLE 1 



Distorted Cartesian Autostereo Hologram 
volumetric volumetric 



Angle perview Ae=— Ae=— AG A0>-^ 

«.v n. A, X 



^ (Ax)- ^ x.Ax 

10 Depth 00 z=^,Az r — '-"T" 
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Claims 

1 . A tliree dimensional video display including: 

- a screen (4) on which successive frames of patterns can be displayed; 

5 - a collimated light source (6) for illuminating the screen (4) in synchronism with tlie 
display of successive frames; 

- tlie arrangement being such that a tliree dimensional moving image can be observed 
by a viewer. 

10 2, The display of claim 1 wherein the light source (6) is an array of light sources that 
can be scaimed repeatedly onto the screen (4) through a series of discrete directions in 
synchronism with the display of successive frames 

3. The display of claim 1 wherein the screen is formed of a plurality of separate areas 
15 each separately addressable sequentially in a field time to collectively form a complete 

frame of a pattern. 

4. The display of claim 3 wherein the image screen (4) includes a front (8) and a rear 
plate (9) containing a layer (13) of a liquid crystal material and a layer (15) of a light 

20 sensitive material operable to change the voltage on the adjacent layer (13) of liquid 
crystal material, a reflective layer (14) between the liquid crystal material (13) and the 
light sensitive material (15), and electrode structures (10, 11) on the imier faces of the 
plates (8, 9) for applying a voltage across the liquid crystal material (13) the electrodes 
(10, 11) forming collectively an array of separately addressable segments in the liquid 

25 crystal layer (13). 

5. The display of claim 3 wherein the screen is formed by an optically addressed 
spatial light modulator having a plurality of separately addressable segments each 
arranged to receive a sub-pattern in a field time. 
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6. The display of any of claims L 2, or 3 and further including means (1) for 
projecting patterns on the image screen (4, 8, 13,). 

7. Tlie display of claim 6 wherein the means for projecting (1) includes means for 
5 projecting a plurality of field patterns in sequence on spatial separate areas (A to P) of 

the image screen or light sensitive layer (1 5) to form a complete frame of an image. 

8. The display of claim 6 wherein the means for projecting includes a digital micro 
mirror device having a matrix of separately addressable pixels collectively providing a 

1 0 pattern or part of a pattern for projection onto the screen (4). 

9. The display of claim 6 wherein the means for projecting includes a cathode ray tube 
video display for projecting an image or a part of an image onto the screen (4). 

15 10. The display of claim 2 wherein collimated light source includes an optical 
arrangement (5) and an array of separate light sources (6) arranged approximately in 
the focal plane of the optical arrangement (5). 

1 1 . The display of claim 1 wherein the light source (6) is laser light emitter. 

20 

12. the display of claim 2 wherein the light sources (6) are capable of emitting at 
different wavelengths. 

13. The display of claim 6 and including a grating (3) between the projecting means 
25 (1) and screen (4). 
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14 A video display for displaying a large image to an observer, comprising a screen 
for displaying patterns, the screen being formed of a plurality of separate areas each 
capable of receiving a segment of a pattern the segments collectively forming a 
complete frame of a pattern; projection means for projecting a segment of a pattern to 
each separate area of the screen in sequence; means for receiving each segment of a 
pattern and forming a complete frame; and means for illuminating the screen with 
coUimated light to display a large area display. 
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Fig.2(a). 

^ Fig.2(b). 
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Fig. 15. 
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